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ABSTRACT 

The quality of redundancy in language usage can he 
examined to determine its effect on communication efficiency. 
Semiotic redundancy, defined as the quantity of prolixity het%#een 
semantic and pragmatic information, has the potential of reducing 
eqoivocatlcm and error and, at the optimal level, provides maximum 
communication efficiency. . Thus, redundancy din coinnunlcation 
interchanges, when joined with redundancy in the human mei^pry, holds 
the key to information transfer between people. .Research into the 
semantics, syntactics, and pragmatics of information interchange 
yields the optimum level of redundancy for tinily efficient 
coonmunication. . (CH) 
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AjBSTRACT 

A thorough examination of redundancy in teirms of its theoretical 
background, functions, meaning, and classification in relation to 
language and coimnunication is reported in this paper. Redundancy was 
classified in accordance^ with the semiotic dimensions of syntactics, 
semantics, and pragmatics, and explored on the basis of semiotic 
redundancy, process-memory redundancy and between-channel redundancy. 
Semiotic redundancy was defined as the redundancy between semantic 
and pragmatic information; process-memory redundancy as the redundancy 
between the information that was being processed and the information 
prestored in the central nervous system; and between-channel redundancy 
as the- joint information between any two channels. 

Discussion focussed upon (1) the functions and drawbacks of 
redundancy in reducing equivocation and error; (2) the derivation of 
between-channel redundancy and semantic redundancy; (3) the theori- 

r 

zation of process-memory redundancy, semiotic redundancy, and dimen- 
sional redundancy and other forms of redundancy; (4) the optimal 
level of redundancy, subject to the noi-se and environmental conditions 
and individual subjects, so as to achieve maximized communication 
efficiency. Contention was made that process-memory redundancy and 
semiotic redundancy hold the key to human communication, and hence, 
warrant further systematic work. 



A THEOiUilTICAL EXPLORATION OF REDUNDANCY 

IN HUMAN COMMUNICATION 
H. J. Hsia, 

Texas Tech University 
Men are immersed in a sea of information; this is why "man has 
invented dramatically effective codes for handling information" 
(Gerard, 1962, p. 4). The effective coding is for effectiye use and 
control of information. Effective coding and use of information im- 
peratively requires systematic rules; rules imply constraint. Wherever 
there is constraint upon any phenomenon, event, system or language, 
such as the orbit of planets, the color of flowers, the computer 
system, and any written ox' spoken language, there are laws. Lawful- 
ness indicates order, particularly in human communication. Hence, 
constraint is the prerequisite of any law, and "every natural law is 
a constraint," which infers predictability (Ashby, 1956, p. 130). 

Communicatioii is concerned with "human relatedness" (Ruesch and 
Bateson, 1951). Human relatedn.ess implicitly refers to rules of 
communication systems, subscribed to by members of any given community. 
Without grammatic rules, for example, language would be simply chaotic 
and incomprehensible. Rules imply constraint, and every constraint 
constitutes redundancy, and for Ashby (1956) constraint is in fact 
redundancy. Manipulation of redundancy is fundamental in human communi 
cation. 

Redundancy has many interpretations. For classical information 
theorists Shannon, Fano (1949), Cherry, and many others, redundancy is 
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a mathematical concept referring to the proportion of relative entropy. 
On the other hand, for example, "redundancy ... implies an overdetermina- 
tion of meaning" (Meier, 1962, p. 123) and "redundancy is the repetition 
of a 'signal (Smith, 1966, p. 365) ~ the latter is conceivably an over- 
simplification. 

Redundancy in informatioa theory is conventionally defined as one 
minus the relative entropy (relative information) in the following for- 
mula as customarily given: 

Redundancy = 1 - (H / H ) 

actual maximum 

where relative entropy is the ratio of the actual to the hypothetical 

maximum entropy, which is derived from the equally probable occurrence 

of each event, symbol, or unit within a given set. It may be recalled 

that H , is the fundamental concept of information theory, i.e., 
actual 

H = log2 Z± whereas Hj^ja^ = log2^* Weaver says, redundancy generally 
. . . is the fraction of the structure of the message which is deter- 
mined not by the free choice of the sender, but rather by the accepted 
statistical rules governing the use of the syinbols in question. It is 
sensibly called redundancy, for this fraction of the message is redun- 
dant in something close to the ordinary sense; that is to say, this 
fraction of the message is unnecessary (hence repetitive or redundant) 
in the sense that if it' were missing the message would still be es- 
sentially complete or at least could be completed (Shannon and Weaver, 
1949, p. 104>. 

It would be ideal if redundancy could be eliminated and if infor- 
mation processing with any physical channel maximized to the. limit of 
capacity, thereby mininiizing the effort and cost involved in informa- 
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tion processing. It is, however, seldom the case, as evidenced in 
the occtirrence of error and equivocation. A redundancy-free system 
in reality is possible only if the entire channel system of communication 
is completely error-free and has an unlimited capacity. A redundancy- 
free system is entirely feasible and can be worked out precisely in any 
physical communication system by devising the most efficient code. 
A series of error-free systems is theoretically possible but pract-ically 
unrealizable, as exemplified in the error-detecting systems built into 
the more sophisticated computers. The checking system in a computer 
(for example, the use of a binary digit to govern six binary digits 
and check the sum to determine whether an error has been made) is a 
redundant system. Due to the noise-congested channel system and the 
error-prone nervous system of man, redundancy in fact proves to be a 
boon to any communication system connected with a living organism. 
A computer instruction code may be designed to be redundancy-free, 
but a telephone conversation is by no means completely error-free. 
The implication .is that there must be redundancy in any human communi- 
cation system. 

If the analogy is to be carried further, then life in general 
follows a redundant pattern, seeking an equilibrium of about 50 percent 
redundancy — an equilibrium between the new (unexpected) and the old 
(predictable) — between disorganization and organization. Colby, 
(1958) argued that "the formulation of value systems is just one 
aspect of the formulation of redundancy systems," as values provide a 
kind of programming or a code of conduct for the individual. 



Hsia 

4 

Controversial Meanings of Redundancy 

"Redundancy is a property of languages, codes and sign systems 
which arises from a superfluity of rules, and which facilitates communi- 
cation in spite of all the factors of uncertainty acting against it . . . 
and redundancy is built into the "structural forms of different languages 
in diverse ways. No general laws exist" (Cherry, 1957, pp.18-19 and 
118). Redundancy also exists in a variety of forms in connection with 
a sign or symbol processed by one or more modalities. But the fact 
that no study has yet attempted to classify all forms of redundancy 
may be a major factor contributing to the conflicting findings in 
communication research. The need for clarification and rigorous defini- 
tion of the redundancy concept seems to be a matter of primary impor- 
tance; for not all studies employ identical concepts of redundancy. 

For example, as Attneave (1959) has attested, "Bricker's usage 
of 'redundancy' is somewhat different from mine. I consider 'redundant' 
the information which various stimulus components share with one another, 
whereas for Bricker, information in excess of that necessary for the 
determination of certain specified naming or categorizing response is 
'redundant,' p. 85)." Binder and Wolin (1964) have clarified some 
misconceptions about redundancy. In the discussion. of inconsistency 
with respect to Shannon's sequential association. Garner (1962) equates 
redundancy with contingent uncertainty in terms of distributional and 
correlational constraint. Binder and Wolin (1964) in presenting their 
concepts of univariate, bivariate, multivariate, and Markovian redun- 
dancy, seem to be of the opinion that the discrepancy in the redundancy 
concept arises from the models individual researchers have adopted. 
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Classification Problems of Redundancy 

Redundancy derives from entropy or information; therefore, classi- 
fication of redundancy-^is exac^^'i^fiat^lof entropy. To classify informa- 
tion, it seems necessary and profitable to start from linguistics. As 
Weaver has pointed out, communication seems to involve problems at 
three levels: "Level A: How -accurately can the symbols of communica- 
tion be transmitted (The technical problem); Level B: How precisely 
do the transmitted symbols convey the desired meaning? (The semantic 
problem); and Level C: How effectively do^s the received meaning 
affect conduct in the desired way? (The effectiveness problem) 
(Shannon and Weaver, 1949)." 

♦ ♦ . . The technical problems are concerned with the accuracy of 
information transference from sender to receiver, of sets of symbols 
(written speech) , or of a continuously varying signal (telepiione or 
radio transmission of voice or music) , or of a continuously varying 
two dimensional pattern (television), etc. The semantic problems are 
concerned with the identity, or satisfactorily close approximation, 
in the interpretation of meaning by the receiver, as compared with 
the intended meaning of the sender . . . The effectiveness problems 
are concerned with the success with which the meaning conveyed to 
the receiver leads to the desired conduct on his part (Shannon and 
Weaver, 1949, pp. 96-97). 

These three levels of communication problems may be traced to 
the theoretical foundation laid down by Morris (1946) , who defined 
"syntactics" as the inquiry into "the relations of signs to the 
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ob-jects to which the signs are applicable" (Level B problems), 
and "pragmatics" as the inquiry into "the relation of signs to 
interpreters" (which is less broad than Weaver's Level C problems). 
Level C problems apparently include such prevalent problems as 
attitude change and learning, for after all, learning is a change 
of information, or "the change in performance, associated vith 
practice • . ." (Hovland, 1951, p. 613). 

These levels of linguistic study are included in a general term 
"semiotics" by Morris (1946) . Further pursuance in the history of 
linguistic study indicates that Morris was probably influenced by 
Bertrand Russell. Based upon Gestalt psychology, Russell (1927) 
posed three pre-requisites in linguistic studies: (1) physical 
occurrence of words; (2) the circumstances that lead to the usage of ■ 
a given word; and (3) effects- of words. 

All these levels "concern signs and relations, or rules," and 
they "have not the nature of separate compartments, but overlap one 
another . ." (Cherry, 1957, p. 221). Syntactic problems be 
clearly defined, but not semantic and pragmatic problems which involve 
meaning. Meaning of a sign must be defined in terms of the psychologi- 
cal reactions of those who use the sign (Stevenson, 1944). Since 
meaning of a sign or a word, in addition to conventional definition 
or agreement, involves psychological factors of an individual, the 
solutions for both Level B and C problems are a delicate process. 
No study has yet shown the relationships among syntactic, semantic, 
and pragmatic information in communication processes. 
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Communication from a cybemetist's point of view is to -seek the 
reduction of entropy, or as Norbert Wiener says, "to achieve specific 
antientropic ends" (Wiener, 195A, p» 48;. In any communication, meaning, 
a loosely defined word and concept, is always present • Because of it, 
both encoding and decoding processes are deterministic, determined by 
common usage, rules, and conventions — all of which are constraint. 
I', can also be safely assumed that the transformation of events into 
signs in the encoding process is a process of entropy reduction. 
Entropy reduction implies redundancy. In the decoding process, a 
reversal of the encoding process takes place; the receiver concocts 
his meaning based upon signs or signals transmitted by the sender. 

Following this line of reasoning, it seems that there are three 
kinds of information, syntactic, semantic, and pragmatic, involved in 
any communication. There must also be three kinds of redundancy. 
However, information theory is based on the statistical probability 
of the occurrence of signs which, in accordance with experiments by 
Shannon and other information theorists, is entirely within the realm 
of syntactics. Once any researcher ventures into the domain of 
semantics and pragmatics, all kinds of unexpected difficulties emerge. 
Judging from existing studies, it may be safe to assume that there 
is a lack of any precise measurement of semantic information. The 
Bar-Hillel (1955) information model of semantic content working 
strictly within a theoretical model offers no practical assistance 
to the solution of the quantification of semantic information. The 
semantic differential (Osgood, et al , 1957) measures the conno- 
tative dimensions of concepts, rather than the amount of semantic 
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information of a whole coninunication. Wiener (19A8, 195A) expounded 
the measurability of meaning but no methodology has been developed. 
It seems an un^uitful search for the quantification of semantic and 
pragmatic information based on information theory. 

Semantic and pragmatic information, it seems, would likely remain 
two formidable topics in communication research. An attempt is made 
to explore semantic and pragmatic information and redundancy later on. 
Both semantics and pragmatics are recognized by Carnap (1955) as the 
two fundamental forms of metoing. One of the realistic approaches 
to the meaning problem in general and the redundar.cy problem in par- 
ticular is to begin by specifying all identifiable sources of redundancy. 

Redundancy emerges in many foirms. Based upon the conventional 
redundancy concept and the redundancy formula, sor^e 'basic forms of 
redundancy relevant to this study have been or are yet to be identi- 
fied: 

Between-channel redundancy (BCR) — the redundancy rate of in- 
formation between channels, usually the visual and the auditory channels. 

Dimensional redundancy (DMR) — the redundancy rate between in- 
formation dimensions, for example, both loudness and pitch if desig- 
nated to represent the same information or response or a word such 
as "taxicab." 
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Distributional redundancy (DTR) — the redundancy rate obtained 
from the distributional information based upon the frequency of occur- 
rence of every element within n sign system* 

Sequential redundancy (SR) ~ the redundancy rate obtained from 
the conditional intcrmation which, for example, in a twc-letter message 
is the dependence of the occurrence of one letter upon the other. 

Process -memory redundancy (PMR) — the redundancy rate between 
information w:' / in the memory system and information being processed. 

Semiotic redundancy (SMR) — the redundancy rate between semantic 
and pragmatic information. 

DTR and SR are sometimes called content-redundancy or structural 
redundancy or wi thin-channel redundancy. BCR has been empirically 
explored (Ibiia, 1968 b, 1970), but PMR and SMR have not; therefore, 
both PMR and SHR will be treated more thoroughly in this paper. 

Language Constraint as Redundancy 

The simple redundancy in the form of repelrition is as Cherry 
(1957, p. 185) says, an addition. And simple addition is costly; 

but some ingenious ways can be devised to reduce both cost and 

t 
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satiation. In order to reduce cost, it is essential for communica- 
tion to have selective repetition, instead of repeating the whole 
ensemble of signals or signt. The parity check in computer storage 
is essentially based upon the concept of selective repetition. 
Selective repetition must be, however, aimed at the maximum effici- 
ency and minimum error possibility, i.e., to reach the optimal repe- 
tition level. Take the four binary coded messages "00," "01, ""10," 
and "11" for example: if we intend to use one more bit for selec- 
tive repetition, the optimum solution will be "000," "110," "101," 
and "Oil" on even parity check,^ or "OOl," "100," "010," and "111" 
on odd parity check, of which any single 'error contained would be 
detected. The even or odd parity check examines whether the sum 
of the individual bits is even or odd. 

In order to reduce satiation, a much better way can be found 
to introduce redundancy. For example, "taxicab" is a semantically 
redundant representation of "taxi" or "cab," an easily identifiable 
redundancy in language. English, which is said to be 50 percent 
redundant (Shannon and Weaver, 1949,; Goldman, 1953, p. 45) is, to 
be more exact, about 58 percent redundant (Singh, 1966, p* 20). 
Redundancy in language may be termed "constraint" (Travers, et^ al , 
1964) in a common sense definition. In a more elegant form, language 
redundancy or constraint can be differentiated by two major categorie 
distributional redundancy and sequential redundancy (Garner, 1962, 
p. 65, pp. 214-220). For the English alphabet, if each letter has 
an equal chance of occurrence, then we have log 27 = 4.76; however. 
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the actual probabilities for a, b, c, d, e, f, • • . are .082, .014, 
.028, .038, .131, .029 .... Using the standard H =-Y P. log P. 
x^e have 4.129 bits. The relative uncertainty or entropy can be easily 
obtained 4.129/4.76 = .876. Redundancy is 1 - .876 = .124. This 
redundancy is distributional redundancy, a quite sinpla, and easily 
understood redundancy. 

The other language redundancy is sequential redundancy, con- 
cerning the certainty of one letter or-word dependent upon the previous 
letter or letters (word or words); i.e., the occurrence of a letter 
or word is conditional upon lett 's or words preceding it. By an 
empirical test of a randomly selected sentence of 129 letters for ^ 
a subject to guess one letter after another, the average information 
was found to be 1.93 per letter (Singh, 1966, p. 19). The ratio 
or relative entropy is then 1.93/4*.76 = .405. The sequential redundancy 
is thus .595; consequently the difference between distributional re- 
dundancy and sequential redundancy, .471, can be attributed to se- 
quential redundancy alone. One may also calculate the sequential re-- 
dundancy in accordance with the formulas xvorked out by Garner (1962, 
pp. 216-219). 

Sequential redundancy may be empirically determined or mathemati- 
cally derived; neither can be conveniently done. Adopting a simpler 
concept of constraint, information and redundancy may empirically and 
heuristically be classified in terms of its degree of constraint, with- 
out involving exhaustive calculation to obtain the precise amount of 
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sequential entropy and redundancy. From a practical point of view, 
obtaining sequential redundancy from a large ensemble of information 
is prohibitively time-consuming, xf not entirely impossible, and its 
utility value is also doubtful. Sequential redundancy seems to hold 
the key to effective communication. Without sequential redundancy, 
no error can be detected in a communication, and no teaching and 
learning is possible, for there is no way to correct grammar or any 
thing at all. Sequential redundancy alone warrants continuous systematic 
investigation . 

One of the practical uses of distributional and sequential 
redundancy Is the determination of authorship, topic, structure, 
and time of composition by letter redundancy in English texts (Paisley, 
1966). Similarly, composership, structure and time of composition 
of music may be determined by using the redundancy principle (Moles, 
-1968, p. 27-32). , 
Repetition as Redundancy 

The simplest redundancy is of course repetition — a concept often 
used but misunderstood, as seen from a mother's instruction to a baby. 
There are two basic kinds of repetition as redundancy: serial repeti- 
tion, and repetition by multiple channels (Miller, 1951, pp. 104-105). 
If a mother tells her baby, "no, no, no," the repetition may contain 
emphasis or reinforcement, but generally it is serial redundancy, in- 
tended to convey the message without any mistake. Provided that she 
thinks the oral communication is not sufficient, she shakes her head, 
too, as is generally the case. This is repetition with multiple 
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channels — gesture as visual signal to convey exactly tb-" same idea 
as the oral. 

Repetition generally has two purpo'ses: ■ first, to avoid possible 
errors of the communication in transition and in reception, since 
communication in any channel, human or machine, is subject to noise 
disturbance; and second, to detect error if it exists. As frequently 
seen in military orders, such as "Attack at 1700 hours, repeat 1700 
hours," the simple repetition as redundancy tends to serve both 
purposes. Understandably the time of hour has the least redundancy 
in any language; no mistake in times, dates and figures can be caught 
unless repeated because no language provides redundancy for them. 

Indeed, "the most obvious recipe for reliable transmission in 
the presence of (omnipresent] noise is repetition of the message a 
sufficient nunier of times to ensure reliable reception (Singh, 1966, 
p. 37);" however, repetition is devoid of any new information, and 
must take space and/or time; therefore, it is costly in terms of 
communication economy. Furthermore, continuous repetition eventually 
leads to sensory and mental satiation. It is likely that any stimuli, 
when repeated, tends to be associated with the cognitive process by 
human subjects at first, but that gradually such association is no 
longer elicitable, or becomes exhausted. As Zielske mentioned, the 
amount learned might appear to vary directly with the frequency of 
repetition, but repetition during a shorter interval seemed to have 
a greater effect (Zielske, 1959) . Another function of repetition is 
to establish multiple traces in the memory, as found by Hintzman and 
Block (1971); however, facilitation that could be attributed to 
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repetition seemed to be not *great (Earhard & Fuller ton, 1969). The 
result was satiation, and possible semantic satiation, the gradual 
loss of meaning (Amster, 1964), almost a duplication of a much earlier 
study on semantic saturation (Basette and Warne, 1919). Further repeated 
communication was not only incapable of evoking reaction or association, 
but suppressed it (Jakobovits & Lambert, 1964). Worst of all, repetition 
exhibited as well proactive inhibition effects, though it increased the 
strength of possible memory traces, in the short term memory (Cermak, 
1969). 

. Satiation of coninunication association seems to depend upon un- 
certainty pr entropy of communication. An inverse relationship can be 
said to exist between uncertainty and conmiunication satiation; the 
greater the uncertainty the less satiation, as seen from a reaction 
time study to ascertain "the positive and negative repetition effects 
by'Umilta, Snyder & Snyder (1972). 

Repetition could be made more efficient, as pointed out by Pollack 
(1958), who compared two message procedures, i.e., repetition and net- 
work selection: repetition involved repetition of the exact word; for 
the network selection procedure, three words were spoken in succession, 
the final word being the one to be recorded by the listener. For example 
in the series, "Cargo, Oxcart, and Firefly," each of the succeeding 
Vords reduced the final possibility by one half, hence, the three words 
carry three bits of information. The network selection procedure was 
, found considerably superior to simple repetition. Even when repetition 
was used, accuracy of perception of words in the presence of noise 
was better if the successive repetitions wer^i as independent as possible 
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(Pollack, 1959) • An apparently insoluble paradox seems to arise here: 
since an increase in redundancy is proportional to tine and cost, and 
thi? processing and utilization of the enormous input information would 
be made easier if redundancy can be reduced (Barlow, 1959). The solu- 
tion seems to lie in optimal coding — the striking of a proper balance 
between redundancy and cost in time and effort* 
Between-Channel Redundancy * 

Numerous studies have plunged into the controversy over the sup- 
posed superiority of the auditory, visual, or audiovisual presenta- 
tion each to the others. The audiovisixal simultaneous presentation 
involves a theoretical dispute on the interfering and facilitating 
f^ctprs of one channel to the other. But in any dual or multiple 
channel presentation, there is also information shared by different 
channels, i.e., redundant information; hence, there is also between- 
channel redundancy. Between-channel redundancy refers here to the 
redundancy rate of information between auditory and visual channels. 
Al^. other redundancy previously mentioned is, in a sense, within- 
channel redundancy, or content-redundancy. Between-channel redundancy 
is ^ discrete source of redundancy, independent of other forms of 
redundancy . 

Between-channel redundancy refers to the similarity of informa- 
tion between two channels; it is a bivariate model. Conceivably, 
between-channel redundancy is unity when both visual and auditory 
channel? transmit identical information; conversely, it is zero when 
the vipual and auditory channels emit completely different information. 
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It is fairly certain that other things being equal, mutual facili- 
tation for audiovisual information processing takes place when between-* 
channel redundancy is unity; and interference occurs when it is zero 
(Hsia, 1968 a, ,b, 1971). Given the consideration that subjects might 
be either auditory or visual attender, i.e., they are inclined to be 
more effective with or preferred information from one channel to the 
other (Ingersoll & Di Vesta, 1972); then one channel complements the 
other, such as cuing to facilitate the search tasks (Mudd & McCormick, 
1960) or the improvement of the visual signal detectability by a 
supra-threshold auditory signal (Loveless, Brebner & Hamilton, 1970). 

Between channel or bisensory redundancy also improves performance 
as contrasted to unisensory channel in which BCR is nonexistent, or 
at least no impairment of performance occurs (Loveless et al, 1970). 
Many models and theories have been proposed to explain the phenomena 
and functions of BCR in terms of, among others, the energy-summation 
model for intersensory facilitation (Jfickerson, 1970), or the energy 
integration model (Bernstein, 1970; Bernstein, Rose & Ashe, 1970) 
suggesting the added intensities across modalities based upon probability 
sunnaation but not physiological summation (Loveless, Brebner, & 
Hamilton, 1970); the former referring to a response resulting from the 
joint occurrence of stimuli neither of which will produce a response 
alone, and the latter referring to the probability of detecting the 
double stimulus is greater than that predicted from probability sum- 
mation. They may be categorized, for Qur purpose, as the supplemen- 
tary function of BCR. 
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Both supplementairy and complementary functions of BCR are sup- 
posedly effective and valid only wh6n synchronization of bisensory 
Stimuli of the same class and type is perfect. Any lagging, staggering, 
and even overlapping of bi-channel stimuli may interfere with .orderly 
information processing. 

The redundancy of inform^tipn processed^ by both the auditory and 

♦ 

visual modalities is easily conceptualized; but its computational and 
definitional problems seem to be overwhelming. Most studies on BCR 
depart radically from the information theory model and avoid assessing 
the rate of redundancy between channels. Many experiments use the 
word "redundancy" in a common-sense way and interpret the "redundant 
relation" intuitively. 

Chan, Travers, and Van Mondfrans (1965) found that if nonredun- 
dant information was presented simultaneously through two channels, 
the visual channel was more efficient than the auditory channel; 
but how much "redundant" was not ascertained. That study also sub- 
stantiated the theory that a color-embellished visual presentation 
would disrupt the auditory .channel more than a black-white visual 
presentation. Color apparevitly is of another dimension. Obviously, 
as dimensionality increases, information increases. Besides, different 
colors seem to have different values (Mezel, 1958; Tikhomirov, 1962). 
When dimensionality increases but does not result in any additional 
information to be learned, the redundancy has, in fact, no value, and 
has only interference effects. 
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Generally, BCR refers to the similarity of information between two 
channels: let us say, auditory and visual. If A and V are the auditory 
and visual sets of signs, all a's and v*s are the elements in the sets, 
we have: ^ 

Auditory = -|^aC^ |a is the auditory signals 

Visual = |v is the visual signs^ 

Clearly, within each channel, there are a number of signals or signs, 
i.e., . 

A = a^t i = 1, 2, , m, 

V = Vjt j = 1» 2, , n. 

The auditory and visual representations have their information contents 
as customarily defined in accordance with information theory and proba- 
bility theory", i.e., 

Y P (a^) = 1, and ]^ p (vj) = 1, 
and the amount of information is defined 

H (A^ = -Xp^^i) log P (^i^ J auditory information, 
H (V) = - X p(v ) log p (v.); visual information. 

Apparently the between-channel redundancy cannot be the sum of the 
redundancy of the two channels; rather it is based upon the joint infor- 
mation derived from the joint probability. Thus 

I P (a^* ^j^ " 

and the "j^oint information transmitted by both the auditory and visual 
channels is 

II (A,V) = - X p (a^, v^) log p (a^, Vj) 



Hsia f 

if 

19 

If A and V are identical, i»e#, whenever appears, Vj also' occurs, 
then 

H' {A,V) log m = log n , ^ 

Following precisely the same logic applied to the syntactic re- 
dundancy, between -channel redundancy is thus obtained by the conven- 
tional formula 

R = 1 - H (A,y) 
{A,V) 
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Pro cess -Memory Redundancy 

Studies of human memory, based upon a variety of theories, have 
inundated many journals in recent years; findings are mostly contro- 
versial. Among the few things that have been definitely established 
is the existence of pre-stored information. Irrespective of whose 
learning theory one subscribes to, be it Thorndike's, Hull's, Skinner's 
Guthrie's, Kohler's, Lewin's, Tolman's or Wertheimer's, one must assume 
first of all, somewhere in the black box, the existence of stored in- 
formation. Without going into neural transmission, localization of 
memory, and neural .Integration of information, etc., suffice it to 
presume, for our purpose, that information retrieval is possible 
because of the pre-existent information in the memory systems. 
Human behavior is dictated by experience, cognition, and to a certain 
extent, instinct,, all of which can be said to be determined by the 
prestored information; though whether instinct is solely activated by 
information 'retrievable from the memory is a debatable topic. Then 
it is reasonable to assume that redundancy exists as well between 
information in the memory (internal information) , and external in- 
formation, that is, inforrj^tion not in the memory. For example, for 
most Americans the Latin language has a very low redundancy with their 
memo^'y; Chinese a zero redundancy. 

The difficulty level of information processing task — learning, 
searching, and even thinking — is generally determined by the amount 
of both external and internal entropy information. It is then readily 
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seen that difficulty levels are positively related to entropy and 
inversely related to redundancy. The manipulation of redundancy, 
along this line of reasoning, holds the key to information pro- 
cessing, learning,' and very probably intelligence as well. 

This redundancy, the redundancy between information being pro- 
cessed and .information within the memory system, probably the most 

* 

important of all redundancies, may conveniently be termed Process 
Memory Redundancy (PMR). PMR is not exactly "internal redundancy," 
as explained by Brown (1959), "Assuming that when a memory trace is 
established, it usually has some internal redundancy. In other 

- words, the trace is established with more features than are necessary 
to represent the information which the trace is required to store." 
This redundancy. Brown argues, takes up memory storage space, but 
"the trace system (redundancy) can often supply some of the informa- 
tion lost through decay of an individual trace (p. 734)." PMR is 
not the redundancy aaaong various kinds of information stored in the 
memory, nor is it the redundancy between internal information and 
external information which is infinite; it is simply how redundant 
the information being processed is in relation to internal informa- 

-^tion. A high school student who has never been exposed to calculus 
would have extreme difficulty in solving an integral problem, for 
the PMR in this particular case is almost zero; however, a college 
math major who is familiar with calculus would have no difficulty 
in obtaining its solution; the PMR is high so far as he is concerned. 
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The increase of PMR, it may be intuitively conceptualized, 
lessens the difficulty and increases the comprehensibility of a com- 
munication, and used in conjunction with internal redundancy, it may 
ease the canalization in memory, "Canalization is the term used to 
refer to the fact that last connections are built up in the form of 
traces of past experience" (Thome, 1955, p. 313). Assuming the 
existence of random effects of noise within the nervous system, 
the protection of information from loss and interference calls for 
some internal redundancy; but storage economy must 'be taken ^^'^o 
consideration as "sensory information has to be transmitted frow 
place to place in the central nervous system and the reduction of 
redundancy before this is done would enable tlie number of internal 
connecting fibres to be reduced" (Barlow, 1959). Optimal coding 
is then the most important task in communication. 

Some explanation might be in order here: Stored information 
-in the CNS must be organized; organization implies the existence 
of rules or systems, hence, internal redundancy. When the incoming 
information has an almost zero PMR rate, it is difficult to integrate 
with existing information; if it is completely redundant, no inte- 
gration will take place, either. Therefore, there must exist an 
optimum rate of PMR. Presumably the^ optimum PMR is what canalization 
needs. But the optimal PMR primarily depends upon coding (Garner, 
1970) to provide more stimulus features in transmitted information 
so as* to overcome disruption or noise (Bourne & Haygood, 1959) • 

To further propound the PMR principle, one might recall what 
MUller said in the nineteenth century: "External agencies can give 

) 
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rise to no kind of sensation which cannot also be produced by internal 
causes, exciting changes in the condition of nerves" (Muller, r*n4, 
p. 33). Muller jLs talking about the potentiality of the visual and 
audito'^ry reception, and the existence of stored information, but we 
are primarily concerned with the redundancy ratio between the existing 
information in the central nervous system and the information being 
fed into it. To a certain extent, PMR is like the psychological and 
potential similarity Mallach (1958) has expounded. Even in pattern 
recognition, for example, the identifica ton of siitiple, incomplete 
letters of B, C, and 0, ia what is known as "closure" (Engel, Kollat 
& Blackwell, 1968, p. 92) ?^tilizes the PMR principle. 

How to guess missing letters in a text is one of the applications 
cf the redundancy of letters. In Shannon* s "guessing ga&is," the subject 
is to guess the next letter until the right letter is identified. On 
the surface, both are based upon sequential redundancy; however, the 
guessing game and replacement of deleted letters involves PMR. There 
must be information in the memory for subjects to guess the next 
letters 9 or fill out the missing letters and words. 

Results of many experiments may be fitted into the PMR concept: 
the more frequent words have higher thresholds and are therefore 
more easily processed (Solomon and Howes, 1951); this is true even 
under noise conditions. Reading rates are faster with more familiar 
words (Pierce and Karlin, 1957) and the number of repetitions re- 
quired for correct identification is also affected by word frequency 
(Postman and Rosenzweig, 1957) as we know very well by now both word 
frequency and familiarity are associated with PMR. 



Hsia 

24 

Experiments using the redundancy principle but not so specified 
are numerous; for example, the study of "familiarity of letter se- 
quences and tachistoscopic identification" (Miller, Brunar, and Post- 
man, 1954), the results of which show the perception of groups of 
letters is affected by the degree of resemblance of the groups of 
letters to ordinary English. All guessing games, replacement of 
missing letters or words, (for example, McLeod & Anderson, 1966), 
deletion (DeRosa, 1969), "Taylor's Cloze procedurr " paired associate 
learning and most of verbal learning belong to this category. In 
fact, all communication tasks contain a certain degree of redundancy. 
The manipulation of various forms and properties of redundancy, to 
a great extent, determines the efficiency of communication. 

Another implicit form of redundancy is readability. Klare (1963) 
has compiled nearly forty readability formulas, and discussed their 
features, and Watkins (1971) has reviewed many readability formulas. 
All these readability formulas are derived from variations of re- 
dundancy or constraint; they could be summarized on the basis re- 
dundancy formula, i.e., R = 1 « H / H So far as individual 

actual max. 

differences such as intelligence or vocabulary are concerned, PMR 
is involved; the difference can be expressed in terms of prestored 
information. The rationale for using redundancy as the standard 
formula for all readability formulas seems quite legitimate, as 
readability is concerned mainly with distributional information with 
syllables, phenemes. letters, words, and/or the ratio of any combina- 
tion of these four. When redundancy is used to replace readability 
formulas, not only distributional but also sequential redundancy can 



Hsia 

25 

be known* Thus, ' ore precise measure of readability can be made 
available — one which would not be limited only to a particular 
language* 

Semiotic Redundancy 

Communication processes have been diagrammed, flowcharted, and 
categorized in a variety of ways. Basically, communication involves 
three levels of problems, as described by Weaver, the "technical, 
semantic, and effectiveness problems" (Shannon and Weaver, 1949). 
Morris (1946) has delineated communication problems within a semiotic 
universe of which there are. three aspects or dimensions of coimnuni- 
cation: syntactic, semantic, and pragmatic. Briefly, syntactics is 
concernedvwith relations between signs, basically a technical problem; 
semantics with relations between signs and their designata, a problem 
with meaning; and pragmatics with aspects involving sign users, an 
effectiveness problem (Cherry, 1957, p- 241; Shannon & Weaver, 1963 
(1949) p. 4). Syntactic information, obviously, refers to the physical 
properties transmitted or received, i.e., signs and signals without 
intrinsic meaning. Meaning or signification is attached to or derived 
from it in the encoding and decoding processes, both of which are more 
deterministic than probabilistic, depending on usage, culture and en- 
vironments. In the encoding processes the encoder transfers the objects, 
events or thoughts from sources into signs. This involves fundamentally 
two levels of signification: for our purposes these are defined as 
intended and expected information. Intended information is the signi- 
fication the communicator assigns to his transmitted signs, whereas 
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expected information is the signification he expects the receiver ' 
to derive from them. In the decoding processes, a reversal of the 
encoding processes involves also two levels of signification, i.e., 
interpreted and envisaged. The receiver construes his own inter- 
preted information based upon the syntactic information through 
whichever medium, and envisages partly or wholly the signification 
intended by the communicator. The communication processes may be 
represented by the schematic diagram as follows: 
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Figure 1 Schematic Diagram of Communication Processes 
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Letting intended information be I-j^, expected information 1^^ 

syntactic information I^, interpreted information I^, and envisaged 

information I^, in the idealized communication Ij^, and 

should be identical. The encoder endeavors to make 1^^, and 

as similar as possible, and the decoder attempts the same with 

lo, I/, and I^. The link between the encoder and decoder, as can be 
3* 4* 5 

readily seen, is nothing more than the syntactic information trans- 
mitted and received* Conceivably, the ideal relationship among the 
five is perfect redundancy; however, only a partial redundancy can 
be achieved in any real communication situation. 

Considering the simplistic example of a child's crying "ma,ma. 

if 

syntactically it is nothing more than the word "ma, ma," biit it is 
full of intended and expected information. Only the mother is capable 
of interpreting it at a given time under a given situation; it may 
mean a number of things, such as: "I want a toy," "My diaper needs 
changing," etc. When the mother receives the information, she is 
not completely free to interpret the meaning of her child's cry. 
From past experiences (prestored information having probability associ- 
ated with all possible events, e.g., "the child is hungry," etc.), 
she immediately forms her interpreted information based upon the en- 
visaged information which is derived from the syntactic information. 
When these five levels of information are matched, i.e., they share 
a high degree of redundancy, the child's need is satisfied because 
his mother correctly interprets his intended and expected communica- 
tion, and takes appropriate action based on the information received. 
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A new mother has-- a world of uncertainty about what the child's 
cry means; however, from experience by trial and error to relate 
communication with events; i.e., to establish redundancy, she gradu- 
ally, perhaps unconsciously, works out a redundancy system in the 
communication between herself and her child. Only when she becomes 
an experienced mother, is she able to differentiate communication, 
based upon her redundancy system. Similarly, the child is also 
gradually conditioned to discriminate. By a continuous information 
transition, playing the game of semantics and pragmatics, the child 
and the mother will make perfect information transference possible. 
As time goes on, information accumulates in the memory system; thus, 
more sophisticated information transference is possible. Provided 
that syntactic information suffers no change in the transaction between 
transmission and reception, with which most of information theories 
books (Shannon & Weaver, 1949; Goldman, 1953; Edwards. 1964; Pierce, 
1961, among others) dealt, the redundancy between Ij^, and I^, 
I^, is semiotic redundancy. When a Russian told an American about 
"democracy," the American could hardly believe his ears, and invariably 
doubted that there could be democracy under a totalitarian regime; 
likewise, an American could hardly convince a Russian that the U. S. A. 
is a democratic country under a capitalist system. This may be called 
semiotic discrepancy — very low in semiotic redundancy. 
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Syntactic Redundancy, 

Syntactic redundancy may be distinguished as falling into two 
major categories in terms of channel, i.e., within-channel and between- 
channel redundancy. Under each category, there may be found three 
major sub-categories; repetitional, distributional and sequential 
redundancy. Of the repetitional redundancy, further subdivision is 
possible, for example, immediate repetition, partial repetition, 
intervalled repetition, delayed repetition, and any combination of 
the four. Distributional redundancy is what is conventionally called 
structural redundancy, whereas sequential redundancy is simply cor- 
relation redundancy based on the conditional information (See Garner, 
1962) . 

Each repetition of a whole set of symbols or signs may be defined 
as 1, regardless of its actual content. This definition applies to 
immediate, intervalled and delayed repetition because each reproduces 
exactly the same set of symbols or signs; the only variable among 
these three is time. Immediate repetition is the repetition of each 
smallest unit within a set, with no intervening time. Intervalled 
repetition is the breakdown of the whole set into segments which are 
then repeated. Delayed repetition is the repetition of the whole set 
of symbols or signs after a time delay of any duration. 

When a message is repeated over and over, the repetitional re- 
dundancy (but not the within-channel redundancy) is additive, with 
the restriction of 0 < R < N where N is the number of channels and/or 
repetitions. However, all other redundancies are still subject to 
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the restriction of 0 < R < 1. The approaching of unity by PMR suggests 
the gradual "absorption of information, until there is no more informa- 
tion in the message to be processed or learned. Implicitly, repetition, 
which we have explored in the previous section, is the least effective 
redundancy method in learning and communication, when cost in terms of 
time and effort is taken into consideration. 

The obvious strategy of using repetition method as redundancy is 
to adopt a measure of selective repetition, instead of repeating the 
whole thing in order to (1) economize time and space needed for the 
redundant information; (2) attempt to establish the syntactic informa- 
tion with semantic and pragmatic information more easily; In other 
words, to achieve maximum semiotic redundancy, and (3) provide means 
for increasing memory strength so as to achieve maximum retention. 
Understandably, selective repetition must be worked out in view of 
the difficulty, novelty, associability , ?.nd relevancy of the communi- 
cation appropriate for a systematic study in the examination of im- 
mediate, delayed, and intervalled repetition in whole or in part. 
So much the better if the classes and types of stimuli can also be 
differentiated in relation to different ages and sexes as well. 

More effective use can be found in sequential redundancy, as 
contrasted to distributive redundancy. Choosing randomly four letters 
of the alphabet. Miller (1958) set up elaborate strings of letters, 
and found that learning increased with the degree of redundancy, 
though the amount of information processed decreased. In a similar 
study, Hogan (1961) generated strings of symbols of four redundancies 
53, 76, 84 and 90 percent respectively, and found that information pro- 
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cessing time decreased with the increase of redundancy, slowly at first 
and rapidly above 76% • 
Semantic Redundancy 

As syntactic information is more or less stationary, constant, 
and tangible, it is therefore measurable, and has been expounded since 
1949 when Shannon published his book. Such is not the case with 
semantic information, though Wiener (1954) insists that meaning is 
measurable and Bar-Hillel and Carnap (1953) actually set up a model 
to demonstrate that semantic information could be quantified, therefore 
is measurable* Since the efforts of these two scholars, no one seems 
to have attempted the working out of a system to measure the amount 
of semantic information. 

An apparent strategy for appraising semantic information seems to 
exhaust the significance in all its shades and nuances within a communi 
cation message. A starting place, for example, might be the categori- 
zation by Ogden and Richards (1923) of the definition of meaning into 
various unspecified groups. Within each diniension (Ogden and Richard's 
groups), there are a number of subdivisions of meaning which may be 
considered variables. Within each variable there may be levels and 
subdivisions. If necessary, further and finer differentiation is 
possible. Using Ogden and Richards' example on the meaning of "beauty, 
the following categorization is attempted: 

The Meaning of Beauty 

A: a. the simple quality of beauty 
b. a specified form 

B: a. an imitation of Nature 

b. a result from successful exploitation of medium 



Hsia 



32 



c. 


the work of genius 


d. 


the revelation of (i) truth, and (ii) the spirit 




of Nature 


e. 


the ideal 


f . 


the universal 


g- 


the typical 


h. 


the production of illusion 


i. 


leading to desirable social effect 


j- 


an expression 


a. 


pleasure 


b. 


emotion 


c. 


a specific emotion 


d. 


involving the processes of empathy 


e. 


vitality 


f . 


synoesthesis 



Ogden and Richards have, in their way of classification, . listed all 
possible significations of the word '^beauty," but by no means is the 
meaning of "beauty" exhausted in their list. Nevertheless, the sys- 
tematic categorization shows that even a very complicated concept such 
as "beauty" may still be exhaustible in its meaning. The meaning of 
'•beauty" as given by Ogden and Richards may be quantified as follows: 



Table 1 Quantification of the Meaning of "Beauty" 
(as given by Ogden and Richards) with Probability 

Variables 

Semantic Original 
Dimension Probability Variables Definition 



P 



A x^, X2 a, b 

B . • • fVii a, b, . . ., j 

C Zj, Zj, . . . ,Zg a, b, . . ., f 
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By using the model of McGill (1954) or Shannon and Weaver (1949), 
the entropy of the meaning of "beauty" may be worked out. For each 
variable and dimension the communicator or receiver attaches a different 
value or weight. The weight can be regarded as the probable occurrence 
of that variable. Thus semantic information for the word beauty can 
be obtained by: 

H(sem) = -Yp^^5^ 1^8 P^^-) 'Ipiv.) log P(y-) "Ypi^J log P(z ) 

^3 J *^ k 

Considering only the dimensionality of semantic information — an 
analogy of changing the amount of information calculated from letters 
to that of words — semantic infoirmation is then reduced to: 
H'(sem) = - $>(D) log p (D) 

It can be verified that when only the dimensionality is considered 
(all elements in a dimension are aggregated Into a dimensional total) » 
information is then decreased to a great extent. In the case of an 
article, message, book, etc., consisting of many different concepts 
(or essential ideas), each concept can be treated separately from its 
information content in the same manner. However, it is prohibitive 
in the tasks of categorization and computation. 

Current studies, as contrasted to Morris and Bar-Hillel & Carnap, 
were concentrated on semantic similarity and its effects of interfer- 
ence and facilitation. Analyzed in conjunction with acoustic similarity, 
semantic similarity was found to have facilitated total recall (Craik 
& Levey, 1970), or to have interfered with the judgment on two words 
with different meanings (Schaeffer & Wallace, 1970 a) and such inter- 
ference was a function of obligatory memory processes (Schaeffer & 
Wallace, 1970 b). 
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Dimensional Redundancy 

Dimensionality is another form of redundancy, explored exten- 
sively by Gamer (1961) and Hsia (1971). Generally, studies of 
dimensionality were primarily examination of nonverbal or non-language 
elements such as perception of size, shape, brightness in visual com- 
munication, loudness, idtch and duration in auditory communication, 
degrees of odor in olfactory communication and taste of saltiness 
in gastronomic communication. They were absolute judgment or dis- 
crimination tests; for exaitple, Slak's study (1969) on multidimen- 
sional information processing, Biederman and Checkosky's study 
(1970) on size and brightness. 

Dimensionality increases information, but usually not to the 
sum of information from each and every dimension, despite the fact 
that its theoretical upper limit is. When dimension increases, par- 
ticularly when dimensional redundancy is high, dimensionality tends 
to provide cues or clues so as to facilitate infoxrmation processing; 
when dimensional redundancy is low, information increases, and hence 
information processing difficulty increases as well. So long as the 
sum of information based upon all dimensions does not exceed the 
channel capacity, information processing might be easier with the 
number of dimensions at a decreasing rate (Slak, 1969). 

The most difficult task in determining dimensionality lies in 
semantic relevancy and associability, as contrasted to pure physical 
properties such as size, shape, and loudness. Generally, when 



relevancy increases, i.e., dimensional redundancy increases, per- 
formance improves; when irrelevancy increases, i.e., dimensional 
redundancy is low or nonexistent, then information processing suffers^ 

Conceptually, manipulation of dimensionality of information is 
the roost economic way of increasing information processing efficiency. 
In practice, however, not too roany studies have been designed to 
tackle this problero, particularly the semantics of languages. Many 
studies associated stiroulus relevancy with redundancy; however, no 
dimensional redundancy was defined, nor its redundancy rate known. 
The function cf dimensional redundancy seemed similar to other 
redundancies, for example, in the study of Bourne and Haygood (1959, 
1961) dimensional redundancy was found to improve performance and 
facilitate information processing particularly when irrelevant in- 
formation increased. Similar findings were reported frequently, for 
example, studies such as DiVesta and Ingersoll^s (1969), and Bieder- 
roan and Checkosky (1970) • 
Pragmatic Redundancy 

Communication is concerned with pragmatic information toward 
which all syntactics and semantic information strive to achieve their 
ultimate objective, provided that we assume all communications were 
purposive; in other woids, pragmatic information is the end product 
of semantic l .d syntactic information. Our literature search reveals 
no mathematical theory on pragmatic information to have been formulated, 
and studies on "the effectiveness problero" are roostly concerned with 
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attitude change or learning. Camap and Bar-Hillel (1956) set up 
a semantic model based upon probability, but the model is hardly 
applicable to the real world. The pragmatic model of communications, 
in a broader sense, is an attempt to evaluate "all questions of value 
or usefulness of a message, all sign recognition and interpretation 
and all other aspects psychological in character (Cherry, 1957). 
It is even more than that: pragmatic information includes the 
mechanism that triggers reaction from the receiver, elicits his 
response, changes his attitude and opinion, and accumulates informa- 
tion in his central nervous system. The accumulated information is 
what is loosely called "past experience" or "intelligence." Learning 
in terms of its terminal purpose of teaching is concerned with prag- 
matic information, and this is true with any communication in a society 
any methodology or theory which could improve pragmatic information 
processing would result in better communication. This is precisely 
what Shannon and Weaver (1949) call: "The effectiveness problem of 
communication." It is readily seen that pragmatic information must 
consist of both syntactic and semantic information. But the reverse 
is not true, because pragmatic information is the derivation of syn- 
tactic and semantic information; in most cases they are inseparable, 
but can be described in three different spaces for working out an 
acceptable model. 

Pragmatic redundancy can be obtained .by the same procedure as 
semantic redundancy. When pragmatic information is equal to semantic 
information, i.e., perfect semiotic redundancy, it is safe to assume 
that a perfect understanding exists and no errors are possible. 
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However, such equality in reality seldom exists, because one may 
be greater or less than the other* Analogous to the equality, it 
is possible to have both semantic redundancy and pragmatic redun- 
dancy equal zero, similar to "GIGO" in computer language; in this 
case, no understanding is possible. The receiver and the sender 
have completely different or no ideas at all* This is a special 
case where no redundancy exists between semantic and pragmatic 
information in human communications. The difference between semantic 
and pragmatic redundancy and the lack of semantic-pragmatic redundancy 
can be assumed as misunderstanding or misconception. VHien SR « 0 
and PR f 0» it is a trivial case, like the response to the Rorschach 
ink blot. But when SR 0 but PR » J, it is simply a case of the 
sender failing to get the idea across; the communication is destroyed 
by noise or the meaning lost in transference. Suah situations are 
frequent; no receiver can ever get the communicator's ressngc pro- 
cessed entirely. When PR>SR and neither is 0, in a simpl'.^ case, 
it means the communicatee conjures up more than the comnunic^tor 
intends. All implication and inference can be worked out by examining 
the relationship between semantic and pragmatic information in an 
attempt to obtain semantic-pragmatic redundancy. 
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REDUNDANCY FUNCTIONS AND RELATIONS 
Over-Redundancy and Under-Redundancy 

Except for BCR, redundancy is by no means an alltime boon in 
connnunication because of the very likely occurrence of either over- 
redundancy or under-re*!^^j^dancy in any communication situation. 
Over-redundancy may be defined as the rate of redundancy in excess 
of the optimal level, and under-redundancy as that below it. To 
illustrate over-redundancy, we may ask how many readers would reread 
the same*issue of a newspaper a second or third time? When any 
communication is processed a number of times, eventually all informa- 
tion is exhausted because of semiotic redundancy and process-memory 
redundancy. Over-redundancy may be viewed as another form of noise. 
All effects of noise operate in an ovei-redundancy situation as well: 
(1) interfering with new information processing; (2) blocking the 
channel capacity; (3) reducing the normal processing speed; (4) becoming 
increasingly boring and bothersome as the over-redundancy rate in- 



creases; and (5) depriving an organism of the vital information which 
is necessary for its survival. In short, too many cooks spoil the stew, 
and too much redundancy spoils the message, and consequently the 
reception of the communication. Too ra< ch redundancy is as detrimental 
as zero redundancy in a message. 

Under-Redundancy is equally undesirable. There are evidently 
two cases that need to be considered. First, abbreviation which of 
itself is nonredundant_has to be in some way redundant with the user's 
memorized information (interr' redundancy) and its previous defini- 
tion. When memory trace cannc be reestablished or memory storage 
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cannot be accessed the abbreviation is in fact noise, and no more 
redundant presentation of abbreviations will make any difference. 
Random numbers of nonsense syllables which have zero redundancy 
work havoc in any information task. In general, redundancy has 
two opposite functions, working for and against information pro- 
cessing. More accurately speaking, redundancy has two effects, 
facilitation and deterioration. OnJ.y within a certain range is 
redundancy useful. 

Over-redundancy may introduce what is kuo\m as "cue conflicts," 
which affect the subject's certainty (Bruner, 1957). Further evi- 
dence can be seen from Baker and Alluisi's study (1962) in which 
subjects listened to examples of the four types of auditory figures 
to be practiced making identifications of the corresponding visual 
figures. They have found that response time to redundancy figures 
was greater than to random figures, and response time to more complex 
figures was greater than to the simpler figures. 

Generally, appropriate repetition increases redundancy, de- 
creases uncertainty, and facilitates coxnnunication so long as repeti- 
tion still yields new information; of course, the difference between 
input and output becomes increasingly smaller with each repetition. 
As more and more information is processed and stored in the central 
nervous system after each repeated presentation, mere repetition 
can practically exhaust information as PMR reaches unity, i.e., all 
the information that has to be processed is already in the memory. 
By definition, any communication containing no information at all 



Hsia 

40 

is considered noise, for example, a parrot capable of speaking only 
a few words would be very much appreciated by someone who heard him 
for the first time, bjit not for the hundredth time. 

The English language is so redundant that "a fraction of letters 
can be randomly deleted from a reasonably long message without making 
the message unintelligible, 'FOR EXMPLE TONTYIVE PRCET OF HE LITERS 
I TIS SENTENCE HVE BEN DT RANM. (Rapoport, 1965). English prose 
is still intelligible when all vowels are eliminated, but not when the 
difficulty level, or the amount of information, is increased, as in 
this deleted sentence, "probblty a mthmtcl dscpln with ms kn t ths, 
for xmpl, i gmtry r nlytcl mchncs." This is an example of "under- 
redundancy." An extreme case of under-redundancy , for instance, is 
a table of random numbers, which is actually a pure case of zero 
redundancy. It may be seen that an optimal range of redundancy must 
be reached to insure intelligibility. 
Optimal Redundancy 

Optimal redundancy of information is supposed to increase the 
,c-l^*.iency in information processing, retention and transfer with the 
following capabilities: (1) capability of reducing to a tolerable 
level errors in the encoding process, and checking out errors in the 
decoding process with its built-in constraint system; (2) capability 
of reducing the effects of noise, interference and distortion in both 
the external channel and internal (physiological) channel; and (3) 
capability of facilitating association and discrimination, establishing 
memory traces in the organism's central nervous system, and helping 
to prevent it from forgetting. 
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These capabilities exist if, and only if, redundancy does not 
reach the cut-off point beyond which redundant connnunication gradually 
reduces information and finally becomes devoid of any information what- 
soever — "over-redundancy*" There is presumably a critical point 
beyond which no additional redundancy would ever increase communication 
efficiency, but would impair it* Optimal redundancy can only be em- 
pirically derived, subject to individual PMR. Individual PMR is 
almost a mysterious concept - on reading an article for the thousandth 
time, all information is exhausted; however, on listening to Mozart 
for ::he thousandth time there is still abundant information that can 
be associated, interpreted, and therefore, enjoyed. • 

Generally, we can only attempt to explain the rudimentary concept 
of optimal redundancy. In a specially designed experiment of spatial 
signal patterns, the critical point of redundancy has been derived 
(Hsu 1963). A considerable part of the information contained in a 
signal structure, Hsu points out, would become "redundant" in the 
course of frequent repetition of the required operation in association 
with a definite signal. Three experiments involving naming numbers 
and pointing to lights in nine--choice tasks with relative stimulus 
frequencies (Fitts, Peterson and Wolpe, 1963) substantiate Hsu's 
interpretation. As redundancy increases, average reaction time to 
the frequent stimulus components increases. The delay in processing 
time proportionally raises the cost of information processing in 
terms of decline of information processing capacity at a given time. 
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Striving toward optimum redundancy as the most effective use 
of the redundancy concept, many studies varied information relevance, 
dimensionality, time, and speed, in an attempt to arrive at the op- 
timum loading for the subject so as to achieve maximum information 
transfer. As a rule, more information can be transmitted by increasing 
^the dimensionality of input information. A study by Anderson and Fitts, 
replicated by Shinkman (1961) sufficiently demonstrates this concept. 
When dimensionality increases above the discrimination capacity of 
the individual, information gain might suffer. Another important 
factor is time; the length of time is directly proportional to the 
amount of information. Iflien the information processing time is pro- 
longed, it is similar to lessening the information loading, and in 
many instances reducing the information complexity and dimensionality, 
as the information can be differentiated into small segments for 
easy processing. In studying the effects of repetition and spaced 
review upon retention of a complex learning task, Rejmolds and Glaser 
(1964) have found that variation in repetition, a simple manipulation 
of redundancy, has only transitory effects upon retention; but spaced 
review involved with the time factor produces significant facilitation 
of retention of the reviewed materials. It must be noted also that 
spaced review is another form of redundancy. 

Departing from counting words and frequencies of the occurrence 
of input information, many studies have shown a high degree of sophisti- 
cation in manipulation of redundancy experiments. For instance, Bricker 
(1955) arranged patterns of lights in a row of five pairs; only one 
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light of each pair could be lit. With three pairs of lights, the 
maximum uncertainty was 3 bits, but with five pairs of lights the 
maximum uncertainty increased to 5 bits, although only 3 bits of un- 
certainty were ever used. Three conditions of varying amounts of 
information and redundancy were administered. Redundancy in this 
study definitely shows a detrimental effect upon learning, and the 
speed of responding: two redundant sets were learned and responded 
to more slowly even after many trials. The reason is not difficult 
to understand: the stimulus was flashed at .7 sec, and three pairs 
of lights could be discriminated perfectly under all response condi- 
tions; therefore, redundancy is the undesirable property, taking 
away information space or time in orderly information processing. 
Deese (1956) gives statistical backing to his general substantiation 
of Bricker's findings that the more redundant figures require greater 
response time'. But Deese also finds the complex figures are more 
accurately identified at the expense of speed. Commenting on these 
^ two experiments. Garner (1962) summarized, "The extra discriminability 
which comes with redundancy is used, given sufficient time, to pro- 
duce greater accuracy of discrimination, but more time is required to 
make use of the inherently greater discriminability (p. 191)." 

Another pattern discrimination study was conducted by Attneave 
(1955) who generated a series of patterns by placing dots in a two- 
dimensional tnatrix (3 by 4, 4 by 5, or 5 by 7 cells). In the 3 by 4 
matrix the dots were always placed randomly; but in the others, they 
were placed randomly or symmetrically, by mirroring the smaller random 
matrix. Subjects were asked either to reproduce the patterns or to 
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identify them with a learned label. The symmetrical patterns were 
identified slightly more accurately than were the random patterns. 
The result as interpreted by Gamer (1962) is that "the increased 
complexity and discriminability which accompanies increased redun- 
dancy does not necessarily aid accuracy of identification." Fitts, 
et al. (1956, 1957) used various methods of constructing redundant 
visual patterns, basically of random and constrained forms, from 
which eight mirroring and repeating patterns were generated. The 
random figures were discriminated more rapidly than the constrained 

figures. The differences in discriminating mirroring and repeating 

» 

patterns were small. 

The Relationship of Redundancy to Other Information Properties 

As redundancy always increases at the expense of entropy, both 
entropy and redundancy are required to reach an optimum rate and 
maintain an optimum ratio if communication is to have maximum effici- 
ency. The rationale for this seems obvious: as redundancy increases, 
it takes away more and more space which information might otherwise 
occupy. 

Redundancy exists in every language to facilitate information 
transfer. Regardless of what form it may take, it exists usually at 
the expense of information and is, in theory, always inversely related 
to entropy. Its increase invariably brings about a decrease in entropy, 
and raises the cost of information processing. On the other hand, 
the increase in redundancy and the correspondent decrease in entropy 
improves efficiency. The increase in redundancy also curtails the 
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amount of information to be processed, reduces the effects of noise 
and equivocation, and improves dependability • 

Having examined the relations between redundancy and input and 
between output and input, relations of redundancy to other informa- 
tion properties, notably error, equivocation, and recalled information, 
become clear. From the fundamental formula for redundancy, it is 
obvious that redundancy would be unity only when the relative informa- 
tion is zero: actual input information must be zero; consequently 
equivocation must also be zero. Equivocation maintains a positively 
linear relation with input beyond the capacity limit; hence it must 
also be true that equivocation maintains an inverse relation with 
redundancy when maximum theoretical input remains the same. 

The relation of redundancy to output, error, and recalled (joint) 
information is subject to numerous contaminating factors, chiefly 
process-memory redundancy, arid *has not been determined empirically. 
However, it generally maintains a logarithmic relation with recalled 
information, error, and output. As the relations between output and 
input: information and between input and redundancy are known, the 
formulation of the relation of redundancy to other information pro- 
perties can be worked our. Relative information, the ratio of the 
actual input to the theoretically maximum input, determines redundancy. 
Only variation of the actual input in a given communication varies 
redundancy. Due to the capacity limit and the relation between input 
and output, the relation between output and redundancy is logarithmic: 
output and recall information are positively correlated; therefore, 
the relation of recall information to redundancy is similar to that 
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between output and redundancy. To a lesser degree, this relation is 
true also between redundancy and error. 

The relationship between redundancy and cost is an uncomplicated 
one. Other things being equal, the actual information processing rate 
decreases as redundancy increases; therefore cost increases propor- 
tionally with redundancy. On the other hand, input information with 
no redundancy renders information processing extremely difficult, 
if not impossible. Hence there must exist an optimum relationship 
between cost and redundancy, which can be found in the optimum ratio 
between input information and redundancy. 
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CONCLUSION 

The fundamental problem in communication and learning is to 
achieve maximum communication efficiency. Perfect information 

tr 

transmission and reception (perfect communication) is only theoreti- 
cally possible; by perfect communication is meant that in which the 
information being transmitted, processed, and fed back sustains no 
equivocation or error. 

Any purposive communication has an intrinsic objective: it is 
what Weaver calls "the effectiveness problem" (Shannon, 1949, p. 96) 
Its prerequisite is, however, "the technical problem," to obtain 
maximum communication efficiency; the former indicates minimum 
equivocation (information loss) and the latter signifies minimum 
error in information transfer from the communicator to receiver. 

i 

How to arrive at maximum communication efficiency is a crucial 
matter in the study of communication. In any information transfer, 
particularly in human communication, principally because of man's 
limited capacity in information processing together with other 
physio-psychological limitations, both equivocation and error are 
inevitable. The inevitability of equivocation and error demands 
a remedy, and redundancy is probably the most effective means man 
has found to reduce equivocation and error in coirmiunication. 

The function of redundancy is to curtail equivocation; to reduc 
error to a tolerable level in both encoding and decoding processes; 
to lessen the effects of noise, interference, and distortion; to 
facilitate information association and discrimination; and to reduc 
forgetting. But the introduction of redundancy into sign systems 
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and channel systems invariably raises the cost of information pro- 
cessing in terms of time and space, as redundancy has to take ax^;ay 
message space that information might otherwise occupy. In order to 
reduce equivocation and error, it is necessary to increase redundancy; 
but to increase redundancy is to decrease information. This is the 
dilemma of communication. ' 

The existence of equivocation and error renders perfect communi- 
cation impossible. By manipulating redundancy in a message betwcqn 
processing and memory, and between channels, it appears possible to 
achieve relative maximum information transfer, taking into account 
entropy, equivocation, error, and redundancy. In other words, in- 
formation and redundancy must be maintained at an optimal ratio to 
keep equivocation and error at a relative minimum, and to keep the 
cost of information processing to a minimum. The maintenance of an 
optimal ratio between information and redundancy is of fundamental 
importance in communication and education, for it can increase 
"selection power." While "selection power" may not be exactly equal 
to "intellectual power" as Ashby (1956, p. 272) suggests, it may 
very well be the first step toward achieving maximum communication 
efficiency. 

The key to human communication seems to lie in the determina- 
tion of the optimal rate of redundancy. It is fairly simple to 
determine its optimal rate in the syntactic dimension of informa- 
tion on which most information theory studies are focussed. It is 
still conceptually simple co determine between channel redundancy 
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which is the joint information of any two channels; however, it be- 
comes extremely complex when semiotic redundancy and process-memory 
redundancy have to be determined. If misunderstanding in human com- 
munications is viewed as the result of lack of semiotic redundancy, 
and difficulty in individual information processing as lack of 
process-memory redundancy, then it is imperative to study redundancy 
on a systematic basis in order to determine the precise functions 
of both semiotic and process-memory redundancy, by exhaustively 
examining the rate of both redundancies in human communication • 
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